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ABSTRACT: 

This study examines the role of school sub -city districts in determining the 
performance/efficiency of their member schools. The study identifies low and high performing 
schools and sub-city districts using a three- year panel of data on New York City elementary and 
middle schools. The results suggest that districts ‘matter’ to school performance, even when 
they have no revenue raising responsibility. The imphcation is that accountability systems need 
to be designed to recognize the role of school districts, and hold them accountable for their 
performance as well. 
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I. Introduction 



State accountability systems as well as the system written into the reauthorized 
Elementaiy and Secondary Education Act rely on measures of performance to judge how well 
schools are educating their students. While the role of districts in financing schools is well 
known, relatively litde attention has been paid to any other function the district might have in 
determining school performance. Advocates for school-based budgeting and school-based 
financing argue that educational policymaking and primary control over budgeting is best left to 
schools, with more limited responsibilities for districts in areas such as support services for joint 
purchasing or professional development. At the same time, the movements toward greater state 
financing and more stringent state accountability systems are strong forces shifting revenue 
raising and authority over curriculum fiom the district to the state level. Do districts continue to 
matter at all in how schools perform? Why and in what ways? 

New York City provides an excellent laboratory in which to study these questions. New 
York City’s elementary and middle schools are organized into 32 community school districts that 
receive resource allocations fi'om the umbrella citywide school district that is dependent on the 
city government for revenues. State and federal funds, by and large, are channeled through the 
citywide district to the community school districts. Each district has a superintendent in charge 
of operations and instraction and in almost every respect, other than raising revenue, the district 
functions in a manner similar to independent school districts. Thus, the differences in school 
performance across these districts is not due primarily to differences in taxation and/or tax bases, 
but to differences in the effectiveness of schools and districts in dealing with their differing 
populations. This study uses data fiom New York City’s public elementary and middle schools 
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to examine the differences in school performance across school districts and to investigate the 
role of the districts in shaping school performance. 

The chapter is organized as follows. In section II we review the relevant literature on 
districts and accountability and in section III we develop a model of the production of education. 

We describe the data used in the analysis in section IV and we present results on sub-city 
districts in section V and on schools in districts in section VI. In section VH we conclude. 

II. Literature 

In the past decade two-thirds of the states have developed accountability systems with 
most focusing more heavily on school rather than district performance. Of the 33 states that 
have statewide accountability systems, four hold only districts accountable for student 
performance, 13 hold both districts and schools accountable, and 16 place all of the 
accountability at the school level (Goertz, Duffy & LeFloch, 2001). The policy and research 
focus, however, has been on holding schools accountable as illustrated by the 1996 volume on 
accountability entitled. Holding Schools Accountable: Performance-Based Reform in Education 
(Ladd, 1996a). The focus on schools is clear. Ladd writes in the introduction, “[M]any people 
believe that schools should be held more accountable for the academic performance of their 
students (Ladd, 1996b p. vii).” Districts in large part are absent from the volume’s papers, either 
because districts are not part of the school-based programs (e.g., Clotfelter & Ladd, 1996) or 
because the research focuses on schools. In the case of Mississippi, the state relies on relative 
performance categories to hold districts accountable through its accreditation process that has 
low incentives and low sanctions (Elmore, Abelmann & Fuhrman, 1996). It is somewhat 
surprising that districts are not more fiilly considered in accountability systems that involve high 





stakes, such as financial incentives, promotion, or sanctions, given that districts serve at a 
rtiinimijm to coordinate educational services of their schools. Further, this oversight might lead 
to ineffective pohcies and/or sanctioning or rewarding of schools. Consider the possibhity of 
two schools that in every respect may be similar, but diifer in terms of performance. Is an 
accountabdity system fair if it does not include the district, given that the difference in 
performance between the two schools may be attributable to their districts? . 

Research on the production and costs of education has shifted fixrm district- level analyses 
to school- level analyses as school-level data are increasingly available. Furthermore, the school 
is often the unit of analysis most appropriate for production related studies. As shown by 
Hanushek, Rivkin and Taylor (1996), production functions may be biased when aggregated data 
are used and models are mispecified. 

With regard to districts, other than their role in school performance, researchers have 
examined a wide variety of topics including district administrative spending, district 
consohdation and district size. Using a ten- year panel of data on district spending and 
performance in New York State, Brewer (1996) finds that educational performance is lower in 
districts with higher district administrative spending. The difference, however, is not statistically 
significant. School district consohdation is often proposed for smaller districts as a means by 
which they can lower their costs, taking advantage of economies of scale of the larger 
consolidated district. Districts with less than 500 students may benefit, in terms of cost savings, 
fi’om consohdation (Duncombe, Miner & Ruggiero, 1995). In terms of performance, smaller 
school districts moderate the negative relationship between socio-economic factors and student 
performance (Howley, 2000). Using a single cross-section of data on pubhc high schools in 
New Jersey, Fowler and Walberg (1991) find strong relationships between district socio- 



economic status and high school performance. In her study of Ohio’s pubhc schools, Schwartz 
(1999) examines the role of districts in allocating funds across schools and contrasts district- 
based allocation de facto formulas to a simulated statewide funding system. She finds that 
districts do differ in their distribution of spendiiig across schools, and that a statewide system for 
allocating resources would differ substantially fi'om the current system by narrowing disparities 

i 

in school spending pohcies across districts. 

III. Model of Production of Education' 

The factors determining the output of schools and districts can be investigated using a 
production function to measure the maximum amount of output that can be produced fi'om a 
given quantity of inputs. In its general form, the production function can be represented as: 
Q=/(Xi,X2, ...X„) (1) 

where Q represents the quantity of output, Xi, X 2 , ... X„ are the n inputs to production, and / (.) is 
the transformation linking them. Historically, district- level analyses have dominated the 
research literature, largely because school- level data were seldom available across or within 
districts. Conceptually, school-level data are preferable for estimating production functions 
because they more closely represent the actual operation of schools within districts. But the 
choice between school- level and district- level analyses is not so clear-cut in efforts to use 
production function estimates to measure effectiveness. In this case we are interested in both, as 
schools operate within districts.^ As we noted above, districts may well perform important 
functions that go beyond revenue raising, such as estabhshing and coordinating educational 
programs for students and teachers in the district’s schools. 
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Using a three- year school- level panel of data on schools and districts that includes 
measures of school performance as well as average characteristics of students and resources, we 
first estimate the following model of production: 

TSsdt = OCo + OCiTSsdt-l + 0C2STsdt + asSCsdt + 0C4T + ttsD + £sdt ( 2 ) 

Where s indexes schools, d indexes districts and t indexes time and TSsdt is output measured as 
average student achievement on tests. TSsdt- 1 is output one period ago, STsdt is a vector of school 
and student characteristics, SCsdt is a vector of school inputs (purchased or donated) and 
characteristics, T is a vector of time dummies, D is a vector of district dummies, and £sdt is an 
error term (or several error terms) with the usual statistical characteristics. The coefficients on 
the district dummies, D, are known as district fixed effects. These district effects (a 5) measure 
the residual variation in school output unaccounted for by variation in inputs across schools, 
student characteristics, etc. that is systematically related to a given district. Thus, these can be 
viewed as measures of technical efficiency - more specifically, the relative efficiency of the 
districts in the sample - controlling for differences in student and school characteristics.^ 

The district fixed effects capture the impacts of unobservable or unmeasured factors, but 
our data also include some observable information about the districts that might be important 
determinants of school performance. These observable fectors include district size, measured 
both by the number of students and the number of schools; average student performance in the 
district as indicated by test scores; average level of resources; and measures of resource use, such 
as percentages devoted to various fimctions. Estimating the impact of these characteristics on 
school performance can be accomplished in two ways. A two- stage method would first estimate 
the district- effects (a 5 ) in a model like (2) and then estimate a second regression explaining the 
variation in the district effects with the district characteristic variables described above. A 
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second, and statistically more attractive method, includes the district- level variables in the first 
stage model, and estimates the impacts directly/ The estimated coefficients on these district- 
level variables, then, offer insight into what it is about districts that matters in relation to school ' 
productiort Thus, equation 2 is modified as follows: 

TSsdt = oco + ttiTSsdt-i + ocjSTsdt + ttsSCsdt + 0C4T + asDS* + a6DSdt+ ayDCdt+Ssdt ( 3 ) 
where DSdt is the average output of the district, measured as test scores, DTdt is a vector of 
district characteristics, and DCdt is a vector of average school inputs. 

To estimate equations 2 and 3, we use school-level data that provide the commonly-used 
measures of student and school characteristics and use robust standard errors to correct for 
clustering of schools within districts. In terms of outputs, there are multiple measures available 
fi'om which to choose. The New York City Board of Education (BOE) tests students in grades 
three through eight, using citywide or statewide reading and mathematics exams. The two most 
commonly reported measures of outputs are the percent of students meeting a required level, 
often the nationally normed 50**’ percentile, and the average normal curve equivalent (NCE) 
score. The former is much easier to interpret, but the NCE score is preferable because it 
provides more information about the school’s (or district’s) overall performance rather than 
simply the proportion of students meeting a specific cutoff. Because tests and their standard 
deviations change across time, we standardize the scores in each tested grade to the citywide 
average by calculating a Z score,^ which represents the distance between the score for a 
particular grade or particular school and the citywide average, relative to the standard deviation. 

We then average aU the grade-specific Z scores to obtain a school-level average Z score.^ 

IV. Data 
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Sources 



This study uses a three- year (1996-97 through 1998-99) panel of data obtained from two 
main sources, of the New York City Board of Education (BOE). The School Based Expenditure 
Reports (SBER) provide rich and detailed data on school- level spending and a count of the 
number of full-time equivalent teachers in a school. For example, spending can be broken down 
by student type with spending on students who are in special education classroom settings on a 
full-time basis separated from spending on students in general education classrooms and their 
related resource room services. Additionally, the data contain a wealth of information on the 
functional purpose of the expenditures. The broadest categorization of spending breaks down 
expenditures into those that directly provide services to schools, those that provide for district 
administration and those that cover systemwide costs, such as debt service. Within each of these 
categories, there is information on the purpose of the expenditures. For example, direct services 
spending is broken down into more detailed categories such as classroom instruction, 
instmctional support and leadership. Even finer detail on the functional purpose of spending is 
available within these categories. The Annual School Reports (ASR) provide school-level data 
on student and teacher characteristics and student performance. In addition to these two sources, 
the study uses data obtained directly from the BOE to augment measures of school- level student 
performance on citywide reading and mathematics exams, specifically data on individual NCE 
scores that are aggregated to the school level. District- level data are created from these school- 
level sources and are described below. 

Sample Size 

There are over 1,100 public schools operating in New York City. Nearly 900 of these are 
elementary and middle schools organized in 32 community school districts while the rest are 
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high schools or citywide special education schools.^ The number of schools changes from year 
to year for a variety of reasons, reflecting the opening of new schools or the closing or 
reorganization of existing schools. We analyze school performance by school, we include only 
schools with reading and mathematics test results in each of the three years. Of these 892 
schools, we exclude 44 schools with missing data on test scores or missing data on a majority of 
the other analysis variables and two schools in a special district. Thus, a three- year (1997 
through 1999) balanced panel data set includes 846 schools. 

V. Sub -City School Districts in New York City 

Other large urban districts, such as Chicago and Los Angeles, have some form of 
governance stmcture within that of the larger citywide district.* New York City’s unique 
governance stmcture, however, is the most deeply rooted and longest lasting of the 
decentralization models and provides an appropriate model from which to glean insight into the 
role of districts in determining performance. The 32 community school districts that govern 
New York City’s elementary and middle schools in effect operate as districts might in a single 
statewide pubUc finance system. 

New York City came to its decentralized governance stmcture in 1969 when the state 
legislature authorized a radical transformation of the city’s centralized control. Thirty- two 
community school districts were created to govern the elementary and middle schools. Each 
district would have its own locally elected community school board and a superintendent in 
charge of instmction and operations. The central office retained control over high schools, 
systemwide budgetary functions, collective bargaining negotiations, pupil transportation, food 
services, school safety, fringe benefits, legal services, school maintenance and constmction and 
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renovation of schools. Under this decentralized governance structure, the central office allocates 
resources to community school districts through a series of formulae that take into account the 
number of students by grade and regulated or mandated class sizes. The locally elected district 
school board and superintendent are then responsible for developing the district’s budget and 
allocating resources to schools. In 1996, new legislation shifted budgetary and administrative 
power away fi'om the locally elected district school board and to the individual district 
superintendents and the system’s chancellor of schools.^ District superintendents are now solely 
responsible for the district budget and resource allocation decisions. 

Perhaps the most prevailing impression of New York City’s pubUc school system is the 
sheer size: over one milUon students, 1,100 schools and $13 billion in total spending. As is true 
for most large urban school systems. New York City’s pubUc schools enroll veiy high 
percentages of students who are racial or ethnic minorities and who are eUgible for federal fi'ee 
lunch programs, limited English proficient (LEP) and/or recent immigrants to the United States. 
Just as there is diversity across districts within a state, there is diversity across New York City’s 
districts. The diversity is not simply in terms of student demographics, but also in terms of the 
structural organization of the districts themselves, as reflected in the number of students, number 
of schools and the size of those schools. Furthermore, even though each district receives 
operating funds through the central formulas, total spending does vary across districts, as does 
the manner in which districts and schools spend their resources. 

As seen in Table 1, there are nearly 700,000 students enrolled in 846 elementary and 
middle schools in the 32 commxaiity school districts. The average district enrolls 21,815 
students and has 26 schools, 20 of which are elementary schools and 6 middle schools. Of the 
645 elementary schools, 6% (37) enroll 0-300 students, 24% (156) enroll 301-600 students and 
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70% (452) enroll over 600 students. Proportionally, there are far more middle schools in the 
smallest category, 20%, and far fewer in the middle category, 9%. What these averages and 
sums mask, however, is the difference across districts as hinted to by the minimum and 
maximum values reported in the same table. 

Insert Table 1 here 

Consider the two extreme cases in terms of district size: District 1, a geographically small 
district on Manhattan’s lower east side that has seen its school-aged population decrease for 
decades; and District 3 1 that covers Staten Island, a populous borough known for its stable 
middle-class environment. District 1 enrolls 8,536 students compared to District 31’s 38,107. 
Interestingly, fewer schools are not synonymous with lower enrollment, testifying to the 
organizational independence of the community school districts. While District 3 1 has more 
schools (48) than any other district. District 1 does not have the fewest; District 16, located in 
central Brooklyn, has only 14 schools, one of which is a middle school, and just under 10,000 
students. District 4, located in the East Harlem neighborhood of upper Manhattan, is known 
across the country for its success in creating small learning communities. With just over 12,000 
students it has 38 schools, only 10 fewer than the largest district in New York City. 

The implication is that, while the size of schools depends in part on available space, 
districts in large measure do have control over school size. Those districts interested in creating 
smaller learning communities often are able to find suitable locations or break-up existing large 
schools into smaller ones.’*^ Clearly, districts are distinctly different fiom one another in the size 
of their schools. As seen in Table 1, there is at least one district without any small schools 
(minimum value equal to zero) and there are 12 districts without any small elementary or middle 
schools. Conversely, Districts 1 and 4 have nine such schools each. Large schools, on the other 



hand, are abundant and each district has at least one elementary school with over 600 students 
enrolled Districts may vary widely with respect to school size, for example two of the Queens 
districts are quite distinct; District 27, located in southeastern Queens, has 29 elementary schools 
of which 24 enroll more than 600 students; District 26, also in Queens, alternatively has neither 
small nor large schools as most of its elementary schools enroll between 301-600 students. 

The middle panel in Table 1 reports statistics describing the characteristics of the average 
student in the average district and reports the standard deviation rather than the sum of the 
observations reported in the first panel. Nearly 77% of the elementary and middle school 
students in the average district are eligible for fi"ee limch and approximately 1 1% are Asian, 38% 
are black, 38% are Hispanic and 14% are white. Educational needs are, on average, high, as 
15% of the students are LEP, 6% receive resource room services (part-time special education 
services) and another 6% are in special education classroom settings on a full-time basis. Again, 
these district averages mask the diversity across districts and the different challenges each 
district faces addressing the educational needs of its students. 

Demographically, the districts are quite distinct. Those that are geographically close to 
one another tend to be demographically similar, although not uniformly. District 26, as noted 
above as being in northwestern Queens, is on many measures, truly distinct in comparison to any 
other New York City district, with a low of 22.8% of its students being eligible for federal 
programs. The district with the second lowest rate of eligibility, Staten Island’s District 31, has 
nearly double the rate of eligibility with 40.2%. Districts 7 and 9 in the south and central Bronx, 
respectively, have over 93% of their students eligible for fi^e lunch. 

While New York City is often considered the most diverse city in the United States, a 
close examination of the sub- city districts reveals differences in the racial and ethnic makeup of 
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their students. In six of the seven districts in Queens, the percent of Asian students is greater 
than 1 7% and reaches a maximum of 41% in District 26. Upper Manhattan’s District 6, which 
includes the densely populated Washington Heights neighborhood that is home Puerto Ricans 
and a large share of immigrants fiom the Dominican Republic, and most of the districts in the 
Bronx, except the northeastern most district, enroll high percentages of Hispanic students 
(greater than 60%); nearly 90% of District 6’s students are Hispanic, for example. In the central 
Brooklyn districts over 80% of the students are black. These districts encompass neighborhoods, 
such as Fort Greene (District 13), Bedford- Stuyvesant (District 16), Crown Heights (District 17), 
East Flatbush (District 18) and Ocean Hill Brownsville (District 23), that are now weU- 
established Africaiv American and immigrant (primarily Caribbean) neighborhoods of New 
York. Districts with relatively higher rates of Asian students, such as Districts 24, 25, and 26 in 
Queens, and districts with higher percentages of white students, such as Districts 20, 21, 22 in 
the southwestern portion of Brooklyn and District 3 1 in Staten Island, tend to have very low 
percentages of black and Hispanic students. 

District 20 in southwestern Brooklyn and District 30 in Jackson Heights, Queens have the 
highest rate of students who are recent immigrants; approximately 14% of their students have 
been in American schools for three years or less. These two districts also have the lowest 
proportion of female students (around 47.5%). Central Brooklyn’s districts 16, 18, 23 and 32, 
with high percentages of black students, also have the highest percentage of female students 
(approximately 50%). Do these figures suggest that immigration rates are higher for school-aged 
males, reflecting the fact that districts with high proportions of recent immigrants have the 
lowest proportion of female students? Answering this question is beyond the scope of this paper, 
but the question does raise an interesting point for further research. 
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In terms of educational needs, the districts with larger percentages of black students tend 
to have fewer LEP students. In turn, in the largely Hispanic district of Washington Heights in 
upper Manhattan, roughly forty percent of its students are LEP. While the districts with the 
greatest percentage of recent immigrant students. Districts 20 and 30, have LEP rates above 
20%, the representations of recent immigrant and LEP are not synonymous. The need for and 
use of resource room services and lull-time special education classroom settings is also not 
consistent across districts. For example. Districts 1 and 31 have over 9% of their students 
identified as needing resource room services, while District 17 in the Crown Heights 
neighborhood of Brooklyn identifies only 3.6% of its students in such need. In terms of full-time 
special education students. District 2 in Manhattan has the lowest percentage, with only 3.2% of 
its students in full-time special education programs. Conversely, District 7 in the south Bronx 
has 10.3% of its students in full-time special educatioa One of the difficulties in describing the 
differences across districts in the percentages of students receiving resource room and special 
education services is identifying factors to which to attribute the difference. Are there simply 
more students in need of such services in a district in the South Bronx compared to one in mid- 
Manhattan? Or, do the districts have different approaches to identifying and integrating students 
in need of special services? Is the location of appropriate fecilities the driving force? Most likely 
it is a combination of these factors. 

The third panel in Table 1 reports spending per pupil and the maimer in which funds are 
spent. Spending ranges fi’om a low of $7,688 per pupil to a high of $1 1,191 per pupil. Over 
88.5% of the spending provides for the direct services of schools; an average of 2.9% of total 
spending covers central New York City district costs. The breakdown of spending by function 
may provide some insight as to how districts are investing resources. Clearly, not aU districts 
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spend their resources in the same manner. While the average district spends 3.0% on educational 
paraprofessionals, there is at least one district that spends 1.3% and one that spends 4.8%. A 
similar range is evident in professional development spending. 

Rather than detail the high and low districts in each of the spending categories, we 
highlight two districts, District 4 in Manhattan and District 19 in Brooklyn. District 4, which 
spends $9,475 per pupil, spends near the minimum proportion on teachers (41.8%) and 
instmctional support (1 1.8%) and the highest on leadership (8.5%). In contrast. District 19, 
which spends $9,035 per pupil, spends near the higher end on teachers (47.4%) and is the highest 
in terms of educational paraprofessionals (4.8%), but lowest in terms of instmctional support 
(8.5%). Demographically, the two districts both have high rates of free lunch eUgibility, as well 
as similar percentages of minority, LEP and recent immigrant students. Clearly they have made 
different choices in how to use resources. 

VI. Results: Schools in Districts 
Descriptive Statistics 

While the discussion above focused on describing students in the 32 community school 
districts, this section focuses on schools (See Table 2). The distinction between average student 
(Table 1) and average school (Table 2) is straightforward. In the former, an average for each 
district is calculated from school- level data that is weighted by the number of students in each 
school, reflecting the characteristics of and spending on the average student in the district. In the 
latter, an average for each district is calculated from school- level data that is not weighted by the 
number of students in each school, reflecting the perspective of the average school. In both 
cases, we then calculate an average for all districts. The average school in the average district 
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has 43.8% of its test takers passing the reading exam and 46.6% passing mathematics. Over 800 
students are enrolled in the average school and the daily attendance rate is 90%, with 92% of the 
students in the same school in June as in the previous October. As seen by the range in the 
rniriimum and maximum values, the average school differs noticeably across districts. 

Insert Table 2 here 

The districts with the highest performing schools in reading. District 2 in Manhattan and 
Districts 25 and 26 in Queens, are also the highest in mathematics. This holds trae as well at the 
opposite end of the spectrum, with schools in District 5 in Central Harlem and Districts 9 and 12 
in Central Bronx having the lowest passing rates in reading and mathematics. District 26 ’s 
schools stand out as high performing in tenns of passing rates, attendance rates, and percent of 
students who are in the same school in October and June. Districts 23 and 32’s schools, both in 
Central Brooklyn, have the lowest average daily attendance rate and percent of students who are 
in the same school in October and June, respectively. Per pupil spending for the average school, 
including spending on students in special education on a full-time basis, is $9,013 and, excluding 
such students, is $8,061. In tenns of special education, per pupil spending is nearly 3 times that 
of the general education average and the range fiom lowest to highest special education spending 
is nearly $10,000. Giving a proper context to spending on full-time special education students is 
difficult because of the widely different needs of such students. Interestingly, the highest per 
pupil spending on full-time special education students is in District 26’s schools, where the rate 
of special education is quite low. Conversely, the lowest spending is in District 23 ’s schools 
where the special education rate is quite high. Does this association reflect economies of scale 
or some other phenomenon? The answer to this question is beyond the scope of this paper, but 
important nevertheless. 
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Teachers, both in their number and the characteristics that are often touted as proxies for 
quahty, exhibit similarly wide variations across districts. The average school in the average 
district has 6.8 teachers per 100 students, of which 81.8% are licensed, 61.4% have five or more 
years of experience and 62.8% have taught in the same school for two or more years. An 
average of $5,036 is spent on students, excluding expenditures on teachers. Schools in districts 
with lower percentages of fiee lunch ehgible, LEP, black and Hispanic students, such as Districts 
25 and 26 in Queens and District 31 in Staten Island, have the highest percentages of hcensed, 
experienced and stable teachers. Conversely, those districts that on average have schools with 
higher proportions of these types of students have the lowest percentages of experienced, 
hcensed and stable teachers. For example, 64.8% of the teachers in the central Bronx District 
12’s schools are licensed, 53.6% are experienced and 53.4% are stable. These rates are at least 
25 percentage points below those for District 25 ’s schools. 

The distribution of school test scores and resources within districts is an issue that 
researchers often address fi'om an equity perspective. The coefficient of variation is a measure of 
dispersion that is often used to assess whether or not the distribution of objects is equitable.* * 
Standards established by previous research suggest that a coefficient of variation greater than 
0. 10 indicates high disparity (Odden & Picus, 2000). How disparate is the distribution of 

resources within districts? Is there variation in the within-district distributions across districts? 

1 2 

Table 2 reports the average coefficient of variation for New York City’s districts. 

The coefficients on the percent of students passing reading and mathematics are high 
relative to the 0.10 standard, with the coefficient on mathematics shghtiy higher than the one on 
reading. Enrollment also varies within districts, with an average coefficient of 0.39 1 . 
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Attendance rates and the percentage of students in the same school in October and June are the 
most evenly distributed factors across schools within districts and across districts. 

Per pupil spending is, on average, somewhat disparate, with coefficients of variation 
exceeding 0.10 in each case. Teachers, with the exception of those who are licensed, also have 
high dispersion as seen in the average coefficients of variation. In terms of spending and 
teachers, however, not aU districts are alike. Some have much higher coefficients of variation 
than others. For example, the coefficient of variation on spending per pupil (All Students) is 
0.448 in District 4 and 0.090 in District 29. The only observable pattern is that 4 of the 5 
districts with the greatest variation in per pupil spending are in Manhattan (Districts 1, 2, 3, and 
4). Notably, Districts 6 and 12 have the highest coefficients of variation on the distribution of 
licensed teachers, as well being among the districts with the lowest percentages of such teachers. 
Conversely, Districts 26 and 3 1 have the most even distribution of licensed teachers, as well as 
the greatest percentage of such teachers. 

School- level Regressions 

Table 3 shows the results of two regressions, one with average school reading Z scores as 
the dependent variable and the other with average school mathematics Z scores as the dependent 
variable. The regressions include district effects.’^ Because the models include district effects, 
the explains over 90% of the variation in reading and mathematics test scores.’ In both the 
reading and mathematics regressions the district effects are jointly significant at the 1% level. 

Thus, districts do matter. 

Insert Table 3 here 

In general, the estimates from reading and mathematics regressions are quite similar. 

The coefficients on the previous year’s scores are positive and statistically significant, meaning 
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that schools with higher test scores in the previous year do better. As is often the case in such 
estimates, schools with higher percentages of students eligible for fiee lunch have lower test 
scores. The same is tme for those schools with higher percentages of black or Hispanic students. 
In contrast schools with more female students have higher test scores. Counter to expectations, 
the coefficient on LEP is insignificant in the reading regressions, but is significant and negative 
in the mathematics regressions. While the coefficients on teacher resources are insignificantly 
different fi'om zero in the reading regressions, schools with more teachers per 100 students have 
lower mathematics scores, but those with a greater percentage of teachers who are fully licensed 
have higher scores. Whether this reflects a tradeoff being made between class size and 
experience is a worthy topic for future research that is outside the scope of this study. As seen 
by the coefficient on the log of enrollment, larger schools have lower reading and mathematics 
test scores. Schools with higher attendance rates and more students who are in the school in 
October and June do better as well. Test scores, in general, tend to be lower in elementary 
schools compared to middle schools. Over the three years, reading test scores have increased, 
though not significantly. Mathematics scores are lower in both 1998 and 1999 than in 1997. 
District Effects 

Our results show that districts do matter; the fixed effects are jointly significant. The 
district effects fi'om the reading and mathematics regressions described above are summarized in 
Table 4. The mean of the district effects in both reading and mathematics is close to zero, but 
there is quite a bit of variation. Are districts that are effective in producing higher test scores in 
reading the same as those that produce higher test scores in mathematics? The correlation 
between district effects in reading and mathematics is quite strong. The Pearson correlation is 
0.82 as shown in Table 4. Their ordinal rank, fi'om highest performer to lowest performer, is 
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highly correlated as well. The Spearman rank correlation is 0.83. As shown in Figure 1, where 
the districts are arrayed in order of the effect size (low to high), the five districts at the bottom of 
the range have distinguishably lower fixed effects than the next group of districts, but they are 
much closer to one another than the two highest performing districts that far exceed the next 
highest performing district. If instead we arrange districts by their respective borough as done in 
Figure 2, we see that even within boroughs there is variation in the district effects. 

Insert Table 4 here 
Insert Figure I here 
Insert Figure 2 here 

While there are many characteristics of districts that are difficult to measure, such as 
district leadership style, some variables we can measure, such as the size of districts, number of 
schools, total resource level and fimctional purpose of resources. Including these variables in the 
regressions, rather than district dummies, may provide insight into how (or why) districts matter. 
Table 5 reports the coefficients on the district- level variables in the school- level regressions that 
also included school- level variables. Although not shown, in large measure, the signs and 
significance of the school- level coefficients do not change. The one notable exception is the 
coefficient on the percent of Asian students. In the previous regression that coefficient is 
positive and insignificant, but here it becomes negative and significant. The coefficients on the 
district- level variables indicate that some are related to performance of schools. Schools in 
districts that have a higher average test score in the previous year do better. The more middle 
schools a district has, the lower the reading and mathematics scores, significantly for reading and 
not so for math. Spending matters to reading scores, but not to math scores. The greater 
proportion of total spending dedicated to educational paraprofessionals, the higher the math 
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scores. Spending on professional development and leadership also matter in terms of math 
scores. Textbook spending is the only spending that is significantly related to reading scores and 
it is negatively related. 

Some of these findings may be counterintuitive; the greater percentage spent on 
textbooks, the lower the test scores, for example. But on average a district spends less than 1% of 
its resources on textbooks. Perhaps the choice in textbooks matters more than the resources 
spent on textbooks. It is interesting that schools in districts that spend more on educational 
paraprofessionals have higher math scores. Perhaps educational paraprofessionals are more 
effective in assisting in mathematics instmction than in reading instructioa There are, as 
always, multiple potential explanations for these observed relationships. Disentangling them is 
beyond the scope of this paper but certainly worthy of further study in the future. Striking an 
effective balance between resource investment and student achievement is one of basic 
challenges districts face. 

Vn. Conclusions 

Although New York City’s community school districts are not responsible , for taxation, 
still their diversity and distinct identities offer an opportunity to examine whether or not districts 
affect school performance. Our results indicate that districts do indeed matter. Our estimated 
district fixed effects suggest an important role for the districts. Whether that effect is due to 
unobservable, time invariant district characteristics or differences in the measurable 
characteristics of the districts is examined and our results indicate that it is the number of middle 
schools, the level of per pupil spending and the ways that resources are spent that matter. 
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Districts do have a role, though often difficult to measure, in the production of education at the 
school level. 

Accountability systems, whether at the state or federal level, may fail to capture the 
importance of districts in the production of school performance, if districts are not exphcitly 
included in performance estimates. Excluding districts fi'om statewide accountability systems 
does call into question the role of districts, not only in accountability systems but also in finance 
systems that lean toward increasing state sources as a share of revenues. Given the wide 
variation across districts in New York City in terms of their resource use and distribution of 
across the district’s schools, districts will have an important role even in a pubhc finance system 
that draws a much greater share of its revenues fi'om state sources. The evidence fi'om New 
York City’s public elementary and middle schools is that districts do matter and they should be 
part of performance accountabihty systems. If districts play an important role in deterrnining 
the academic performance of their member schools, then implementing accountability systems 
that focus sanctions and rewards on its schools will be less effective than a system that holds 
both accountable. In New York City the chancellor of the public schools system holds the 
community school district superintendents accountable for the performance of their schools and 
students. The chancellor produces District Performance Profiles that are used in the evaluation 
process and in dialogues on district and school improvement (Board of Education of the City of 
New York, 2002). This is but one component of an accountabihty system in New York City that 
does focus on both schools and districts. 
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Endnotes 



* Much of the material on production functions is drawn from other publications by Schwartz and Stiefel, notably, 
“Measuring School Efficiency: Lessons from Economics, Implications for Practice,” 2000 \n Improving Educational 
Productivity, editors, David H. Monk, Herbert J. Walberg, and Margaret Wong, Greenwich, CT, Information Age 
Publishing Inc. 

2 

In fact, there are many levels of nesting: kids in classrooms in schools in districts in states. The policies and 
practices at all of these levels arguably affect student performance, but given the availability of data and the 
sophistication of statistical techniques, it would be prohibitive to conduct an analysis that considers all of these 
levels. For now we think it is crucial to bring the district into the picture. 

^ Note that not all the residual variation is attributed to district effectiveness, rather the error term picks up the 
random variation. 

The estimation procedure must, then account for the difference in the ‘level’ at which the district and school 
variables are measured. As described below, we use robust standard errors to control for clustering of observations 
within districts. See Raudenbusch and Bryk (1986) for more on this. 




^ We weight each grade’s Z score by the respective number of students tested in each grade. This helps to account 
for possible differences in the number of students tested by grades. 

^ High schools are not part of the community school districts, but are run centrally from the New York City Board 
of Education. 

^ Chicago’s public schools are divided into six geographic regions. A regional director oversees a staff that serves 
as a liaison to the central offices, provides support to local schools, coordinates desegregation efforts and enforces 
rules and contracts. In June of 2000, the Los Angeles Unified School Dis trict reorganized the district into 1 1 local 
school districts, with over 50,000 students in each district in kindergarten through 12*^ grades. Unlike the Chicago 
regional directors, Los Angeles’s superintendents do have budgetary and instructional authority. 

^ Locally elected school boards had the authority to hire a superintendent and principals for the district’s schools. 
The new legislation stripped them of this power. 

The chancellor previous to the current one issued regulations concerning the size of schools and the financial 
support and designation of smaller learning communities as schools. 

^ ^ The Coefficient of Variation is calculated by dividing the square root of the variance (standard deviation) by the 
mean value. 

The coefficient of variation on the mean Z scores in reading and mathematics are numerically excessive because 
of the values of the Z scores. It is difficult to interpret these numbers, thus, only the coefficients on the percent 
passing measure are discussed. 

^ The regressions include dummies for missing values for %LEP, %Resource Room, %Special Education, % of 
Teachers Licensed, Experienced, In This School for 2 Plus Years, and % of Students in this school (October & 

June). The coefficients on these variables are not reported in Table 3. 

The regressions reported in Table 3 are not weighted by student counts. Weighted regressions were run and 
produced similar results and, thus, are not reported here. 
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Table 1. Characteristics of Community School Districts* in New York City, 1999. 





Mean 


Min 


Max 


Sum 


Enrollment (October Register) 


21,815 


8,536 


38,107 


.698,069 


Number of Schools 


26 


14 


48 


846 


Elementary Schools 


20 


11 


38 


645 


Size: 0-300 


1 


0 


9 


37 


Size: 301-600 


5 . 


0 


12 


156 


Size: Over 600 


14 


3 


24 


452 


Middle Schools 


6 


1 


15 


201 


Size: 0*300 


1 


0 


15 


40 


Size: 301-600 


1 


0 


3 


18 


Size: Over 600 


4 


0 


10 


143 



Standard 





Mean 


Min 


Max 


Deviation 


% Free Lunch 


76.7 


22.8 


94.8 


16.8 


% Asian 


10.6 


0.8 


41.3 


11.8 


% Black 


38.0 


4.3 


89.3 


27.0 


% Hispanic 


37.9 


7.0 


89.7 


23.6 


% White 


13.5 


0.3 


61.6 


15.8 


% Recent Immigrants 


7.1 


2.1 


14.6 


3.7 


% Female 


48.9 


47.4 


50.4 


0.6 


% Limited English Proficient 


15.0 


2.1 


40.6 


8.4 


% Resource Room 


6.4 


3.6 


9.9 


1.5 


% Special Education 


6.1 


3.2 


10.3 


1.9 


Spending Per Pupil 


$8769 


$7688 


$11,190 


693.5 


Percent of Total Spending 
Direct Services to Schools 


88.5 


87.4 


89.4 


0.5 


Classroom Instruction 


53.2 


49.5 


56.9 


1.9 


Teachers 


45.1 


40.7 


49.2 


2.2 


Educational Paraprofessionals 


3.0 


1.3 


4.8 


0.8 


Textbooks 


0.7 


0.5 


0.9 


0.1 


Librarians & Library Books 


0.1 


0.0 


0.4 


0.1 


Professional Development 


1.7 


0.6 


4.5 


0.8 


Instructional Support 


10.0 


7.9 


13.0 


1.2 


Leadership 


7.3 


6.3 


8.5 


0.5 


District Costs 


2.9 


2.3 


4.2 


0.4 



*There are 32 community school districts. 

** Percentages within categories will not sum to the category total as all breakdowns are not included. 
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Table 2. Characteristics of Schools in Community School Districts* in New York City, 1999, 



Standard 





Mean 


Min 


Max 


Deviation 


Test Scores 
Reading: 










MeanZ ofNCE 


-0,029 


-0.491 


0.835 


0.327 


% Pass 


43,8 


25.1 


81.4 


13,6 


Mathematics: 










MeanZ ofNCE 


-0.026 


-0.509 


0.964 


0.364 


% Pass 


46.6 


27,0 


85,8 


15,2 


School Characteristics 
Enrollment (October Register) 


831 


324 


1371 


223,4 


% Average Daily Attendance 


90.4 


87,2 


94.6 


1,9 


% of Students in This School in Ocotber & June 


92.1 


88,2 


94,8 


1,4 


School Inputs 

Spending Per Pupil (All Students) 


$9,013 


$7,877 


$11,305 


698,7 


Spending Per Pupil (Excluding Full-time Special Education) 


$8,061 


$7,094 


$10,067 


573,0 


Spending Per Pupil (Full-time Special Education Students) 


$23,911 


$19,822 


$29,777 


2528,9 


Teachers per 100 students 


6.8 


5.6 


7.8 


0,6 


% of Teachers, Fully Licensed 


81.8 


64.8 


98.1 


9,0 


% of Teachers, Experienced (5+ years) 


61.4 


52.3 


76.3 


5.6 


% of Teachers, In This School 2+ years 


62.8 


49.9 


72.2 


5.5 


Non-teacher Spending Per Pupil 


$5,036 


$4,354 


$6,646 


533.3 


Coefficient of Variation 
Test Scores 
Reading: 










Mean Z ofNCE 


2,622 


-5.409 


57.287 


11.108 


% Pass 


0,316 


0.076 


0.510 


0.107 


Mathematics: 










Mean Z ofNCE 


-0,740 


-9,759 


6.047 


2.954 


% Pass 


0,338 


0.084 


0.551 


0.102 


School Characteristics 
Enrollment (October Register) 


0,391 


0.220 


0.660 


0.112 


% Average Daily Attendance 


0,022 


0.008 


0.037 


0.006 


% of Students in This School in Ocotber & June 


0,038 


0.016 


0.091 . 


0.015 


School Inputs 

Spending Per Pupil (All Students) 


0,191 


0.091 


0.448 


0.066 


Spending Per Pupil (Excluding Full-time Special Education) 


0,150 


0.067 


0.487 


0.082 


Spending Per Pupil (Full-time Special Education Students) 


0,279 


0.158 


0.875 


0.130 


Teachers per 100 students 


0,175 


0.100 


0.603 


. 0.091 


% of Teachers, Fully Licensed 


0,106 


0.029 


0.195 


0.037 


% of Teachers, Experienced (5+ years) 


0,180 


0.106 


0.306 


0.051 


% of Teachers, In This School 2+ years 


0.177 


0.102 


0.282 


0.047 


Non-teacher Spending Per Pupil 


0.240 


0.111 


0.413 


0.062 



♦There are 32 community school districts. 
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Table 3, Parameter estimates, Reading & Mathematics. School-Level 
Production Functions with District Fixed Effects. (Robust Standard Errors) 





Reading 


Math 


Intercept 


- 1.263 


- 1.383 




(0.239) 


(0,238) 


Lag of Reading 


0.763 


0.758 




' (0.015) 


(0,014) 


% Free Lunch 


- 0.001 


- 0.001 




(0,000) 


(0,000) 


% Asian 


0,000 


0,000 




(0,000) 


(0,000) 


% Black 


- 0.001 


- 0.002 




(0,000) 


(0.000) 


% Hispanic 


- 0.001 


- 0.001 




(0.000) 


(0,000) 


% Recent Immigrants 


0,000 


0,000 




(0.001) 


(0,001) 


% Female 


0.003 


0.002 




(0.001) 


(0.001) 


% Limited English Proficient 


-0.001 


- 0.001 




(0.000) 


(0,000) 


% Resource Room 


- 0.007 


- 0.008 




(0.001) 


(0,001) 


% Special Education 


0,000 


0,000 




(0.001) 


(0,001) 


Log Teachers per 100 students 


-0.022 


- 0.029 




(0.017) 


(0,014) 


% of Teachers, Fully Licensed 


0.001 


0.001 




(0.000) 


(0.000) 


% of Teachers, Experienced (5+ years) 


0.000 


0.000 




(0.000) 


(0.000) 


% of Teachers, In this school 2+ years 


0.000 


0.000 




(0.000) 


(0.000) 


Log Non-teacher Spending Per pupil 


0.004 


0.007 




(0,021) 


(0.021) 


Log Enrollment (October Register) 


- 0.021 


- 0.026 




(0,008) 


(0.008) 


% Average Daily Attendance 


0.014 


0.016 




(0,002) 


(0.002) 


% of Students in this school in Ocotber & June 


0.002 


0.002 




(0,001) 


(0.001) 


Elementary Schools 


- 0.014 


- 0.028 




(0.008) 


(0.008) 


1998 


0.001 


- 0.021 




(0.007) 


(0.007) 


1999 


0.004 


- 0.017 




(0.009) 


(0,008) 


R-Squared 


0.9313 


0.9441 


N 


2538 


2538 


F statistic 


513.12 


635.29 
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Note 1: F-test for joint significance of district effects significant at 1% level 
Note 2: Bold indicates significance at the 10% level or better 

Note 3; Regressions include dummies for missing for%LEP, %Resource Room, %Special 
Education, % of Teachers Licensed, Experienced, In This School for 2 Plus Years, and % of 
Students in this school (October & June). 
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Table 4. Descriptive Statistics and Correlations, Fixed Effects 





N 


Mean 


Min 


Max 


Standard 

Deviation 


Reading 


32 . 


0.0000 


-0.0434 


0.0949 


0.0299 


Mathematics 


32 


0.0012 


-0.0472 


0.0674 


0.0280 


Pearson Correlation 


32 


0.8192 








Spearman Correlation 


32 


0.8325 









Figure 1. Reading, District Effects in Ascending Order. 
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Figure 2. Reading, District Effects by Borough. 
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Table 5. Selected Parameter Estimates, Reading & Mathematics. School-Level 
Production Functions with District-Level Variables (Robust Standard Error). 





Reading 


Math 


Intercept 


1.191 


-1.986 




(1.775) 


(1.921) 


District Enrollment 


0.000 


0.000 




(0.000) 


(0.000) 


District Enrollment Squared 


0.000 


0.000 




(0.000) 


(0.000) 


District Average Z Score 


0.123 


0.088 




(0.030) 


(0.029) 


Number of Elementary Schools 


0.001 


0.000 




(0.002) 


(0.002) 


Number of Middle Schools 


- 0.002 


-0.001 




(0.002) 


(0.002) 


Total Spending Per Pupil 


0.000 


0.000 




(0.000) 


(0.000) 


% of Total Spending, Direct Services 


-0.030 


0.004 




(0.021), 


(0.023) 


% of Total Spending, Classroom Instruction 


0.003 


-0.007 




(0.006) 


(0.007) 


% of Total Spending, Teachers 


-0.004 


0.008 




(0.007) 


(0.008) 


% of Total Spending, Educational Paraprofessionals 


0.006 


0.016 




(0.008) 


(0.010) 


% of Total Spending, Textbooks 


- 0.042 


0.015 




(0.017) 


(0.017) 


% of Total Spending, Librarians & Library Books 


0.020 


-0.004 




(0.030) 


(0.031) 


% of Total Spending, Professional Development 


0.004 


0.018 




(0.009) 


(0.011) 


% of Total Spending, Instructional Support 


-0.010 


-0.001 




(0.006) 


(0.005) 


% of Total Spending, Leadership 


0.001 


0.014 




(0.007) 


(0.008) 


% of Total Spending, District Costs 


-0.017 


0.014 




(0.027) 


(0.031) 


R-Squared 


0.9306 


0.9436 


N 


2538 


2538 


F statistic 


3006.88 


878.72 



Note 1: Bold indicates significance at the 10% level or better 

Note 2: The regression includes the school- level variables from the regression reported in Table 3. 
Only the coefficients on the district -level variables are reported in this table. 

Note 3: Regressions include dummies for missing for %LEP, %Resource Room, % Special Education, 
% of Teachers Licensed, Experienced, In This School for 2 Plus Years, and % of Students in this 
school (October & June) 

Note 4: The difference between significant and insignificant coefficients equal to 0.000 and their 
standard errors also equal to 0.000 are only seen when the figures are reported to the millionths. 
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